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Educational Quality Assessment 



An assessment program vas developed to measure the 
efficiency of educational programs used in the school districts 
throughout Pennsylvania. J?he effectiveness of the school vas 
determined by measuring its product, collective student performance* 
This required a design which takes into account other conditiocs 
vbich appear to contribute to student performance. Implementing the 
model vas carried out in three phases. Phase one included the 
development, field testing and refining of the instruments for each 
of the goals, the procedures for collecting data about students, 
school and community conditions, and computer analytic technigoes for 
determining the relationship of school and community conditions which 
exist with student performance. Phase two concerned itself with the 
collection of information stratified from a state representative and 
phase three is the actual assessment of schools, which is a 
continuing activity begun in 1970. The assessment findings can be 
used to provide a focus for examing individual student test results 
which exist, to measure the effectiveness of curriculum changes and 
to provide objective data to support request for programs and 
equipment. (Author/BB) 
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Pennsylvania's General Assembly in 1963 was preparing to enact legislation 

mandating school district reorganization. Legislative hearings on the bill 

included many arguments for and against requiring reorganization. The expression, 

"quality education," figured prominently in these arguments, VHien witnesses 

were asked to define the phrase, the number of different interpretations 

approached the number of individuals testifying. In consequence, Act 299 

included the following mandate to the State Board of Education, 

'*...to develop an evaluation procedure designed to measure 
objectively the adequacy and efficiency of the educational 
programs offered by the public schools of the Commonwealth'* 
and 

"...to provide each school district with relevant comparative 
data to enable directors and administrators to more readily 
appraise the educational performance and to effectuate with- 
out delay the strengthening of the district's educational 
program. " 

Development of the evaluation procedure required both a philosophy and a 

foundation on which to build, a baseline for measurement. This foundation was 

established with the adoption of the Ten Goals of Quality Education by the State 

Board of Education in 1965. 

Ten Goals of Quality Education 

I. Quality Education should help every child eu:quire the greatest 
possible understanding of himself or herself and appreciation of 
his or her worthiness as a member of society. 

II. Quality education should help every child acquire understanding 
and appreciation of persons belonging to other social, cultural 
and ethnic groups. 

III. Quality Education should help every child acquire, to the fullest 
possible extent, mastery of the basic skills in the use of words 
and numbers. 



IV. Quality Education should help every child acquire a positive 
attitude toward the learning process. 

V. Quality Education should help every child acquire the habits 
and attitudes associated with responsible citizenship. 

VI. Quality Education should help every child acquire good health 
habits and an understanding of the conditions necessary for 
maintaining of physical and emotional well-being. 

VII, Quality Education should give every child opportunity and 

encouragement to be creative in one or more fields of endeavor. 

VIII. Quality Education should help every child understand the oppor- 
tunities open to him or her to prepare for a productive life and 
help each child to take full advfmtage of these opportunities. 

IX. Quality Education should help every child to understand and 

appreciate as much as possible of human achievement in the natural 
sciences, the social sciences and the humanities and the arts. 

X. Quality Education should help every child to prepare for a world 
of rapid change and unforeseeable demands in which continuing 
education throughout adult life should be a normal expectation. 

One of Pennsylvania's major contributions to the assessment concept was the 
shift of emphasis away from the measurement of student achievement in basic skills 
only , to the acceptance, and inclusion, of such vital aspects as self -concept » 
understanding others, interest in learning, citizenship, health habits, creativ- 
ity, vocational cognizance, understanding human accomplishments and preparation 
for a changing world. Since the focus was the educational program offered by 
the school, the school became the unit of assessment but its effectiveness was 
determined by measuring the product, collective student performance. 

Measuring total student performance, which can be realistically related to 
the adequacy of the educational program, requires a design which takes into 
account other conditions which appear to contribute to that performance. Informa- 
tion on forty-four selected factors, including what the student brings with him, 
the particular influence the surrounding community contributes, and those unique 
school characteristics which affect him, was collected from state records, school 
administrators, teachers and the students themselves. 



Implementing the model was carried out in three stages or phases. 

Piiase I included the development, field testing and refining of: 

1. Measuring instruments for each of the ten goals. Examples 
of these instroiments: 

Goal II - Understanding Others - consists of 33 state- 
ments which describe situations where differing others 
interact with tl:e individual. Differences are in items of 
racial, religious and social backgrounds or physical and 
mental attributes. Responses indicate the degree of comfort 
the student reflects in each situation. 

Goal V - Citizenship - consists of 54 items which 
measuve the students willingness to exhibit preferred behavior 
in social situations under a variety of motivating conditions. 
Social contexts are presented by 18 stories. Bach story has 
three items which indicate positive or negative consequences 
resulting from the actions taken. Responses are Yes, Maybe 
or No. Subscales include welfare and dignity of others, 
respect for law and authority, and personal responsibility 
and integrity. 

2. Procedures for collecting data about students, school and 
community conditions. 

School information including such elements as enrollment, 
percentage of attendance, staff-pupil ratio, instructional 
expenditure per average daily membership and geographical 
location of the building was collected from the principal. 

Questionnaires were completed by teachers employed in 
the school which provided data on age, experience ^ level of 
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praparation together with their attitudes toward their careeir» 
their satisfaction with students, parents, fellow teachers 
and administrators a? well as other information useful in 
identifying a school climate. 

Students responded to questions about home backgrounds, 
attitude toward school, accessibility of the school library 
and guidance services and other items found to be helpful in 
establishing the school climate. 
3» Computer analytic techniques for determining the relationships 
which exist with student performance. 

Investigation was conducted with various multiple 
regression techniques and expanded to use the QSASE program 
for finding partial and semlpartial relationships. 
Phase II concerned the collection of information from a state repre- 
sentative stratified, random sample of schools using enrollment size 
and relative school district wealth as evidenced by the aid ratio. 
This information was utilized to establish the frame of reference 
for pupil, school and community conditions and their relationship 
to student performance. This was carried out for fifth and eleventh 
grades in 268 school districts, encompassing 400 schools, 1300 
teachers and 37,000 students. 

Plmse III is the actual assessment of schools. It is a continuing 
activity begun in 1970. To date 302 of Pennsylvania's 505 local school 
districts have participated in this phase. 
Pennsylvania's legislature, emulating Congressional actions, did not provide 

the necessary funds» estimated to be $1,000,000 annually, to enforce the mandate. 

Participation in the assessment program, consequent ly, has been on a voluntary 



basic with the local school district making application to be included in a par- 
ticular year* Participation includes the commitment of the district to name 
a representative or coordinator as well as to provide teacher monitors and maka 
available 4 hours of stude.nt time at the grade to be assessed* 

School district applications are received in September, decision*^ on 
acceptance made in October, contact is established immediately thereat^er with 
the district representative to confirm approximate student enrollment to par- 
ticipate in the fall and other essential planning information. 

Regional workshops are conducted for the district representatives, by 
the division staff in February. The major emphasis of these meetings is to make 
clear that the focus of assessment is upon the school program and not the stu- 
dents, not the teachers or the administrators. Once this concept appears to be 
accepted some rather intensive training is begun on the mechanics of the collec- 
tion of data. This .includes the need to standardize procedures, the importance 
of obtaining complete infonnation necessary for accurate prediction, methods for 
selecting samples of teachers to answer questionnaires and the principles of 
coding information. Representatives are provided with manuals* and sample forms, 
given exercises in coding, selecting teacher samples, packing and shipping of 
various components. The confidentiality aspect is reemphasized - teachers are 
to seal their questionnaires before returning them to their coordinator, teachers 
are requested not to look over students' shoulders or through partially completed 
questionnaires, students are to physically remove their name labels as a step 
in completion of the assessment - and finally, the explanation of the policy to 
make assessment results available only through the local district superintendent. 

* Burson, Willieim W. "Manual for Interpreting Intermediate School Reports - 
Grade 9," Pennsylvania Department of Education, Karrisburg, 1973. 
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Data collection is done the first two weeks of March. Data processing is 
performed during the next three months and the compilation of individual school 
reports requires one more month. In October and November of the year following 
the data collection, approximately six months later, individual school reports 
are delivered to the local school district administrator by teams composed of 
department staff members augmented by specially trained staff from regional 
educational development centers. 

Administrators are requested to set aside one-half day of time to review 
the report with the interpreters, to make certain the coordinator is present 
and to invite any other members of the district staff, board or interested 
citizens he desires. Upon arrival, the team customarily follows a pre-established 
routine of Interpreting one school report - the average level of student per- 
formance on each of the Ten Goals of Quality Education, the school's standing 
in the state on each of the Ten Goals, the school's standing relative to schools 
operating with similar sets of impinging conditions, a comparative analysis on 
each of the different conditions employed in establishing the predicted or 
expectancy level, and aggregated student responses to selected groups of items 
for most of the Ten Goals. Suggestions for comparing findings for additional 
schools within the district, recommendations for possible follow-up procedures 
and the offer to supply additional help, upon request, are part of the initial 
interpreting session. 

The availability of Information to agencies other than the local school 
district participating in the assessment Is restricted for approximately six- 
months until the local school administration has sufficient time to digest and 
publicize the results. State education agencies have easy access to aggregated 
information but are required to secure release from the local school district 
before individual school information is made available. 



Legislative committees and regional organizations ace provided aggregated 
information upon request. Since this is the most useful form for their purposes 
no major conflicts have arisen. If individual school district information i« 
requested it can be obtained directly from the local superintendent or upon the 
presentation of a signed release* 

The U.S. Office of Education receives information, largely in the form of 
identified needs for specific types of schools or school districts, geographical 
locations or regions, community sizes or as aggregations of statewide strengths 
and weaknesses. 

It might be well to mention at this juncture, that the assessment data is 
made available subject to the usual limitations of staff and money, to any 
recognized research orgcmization wishin to use it with the agreement to provide 
Pennsylvania with its findings. 

Such studies are presently underway at Johns Hopkins University, in the 
development of psychosocial maturity scales; The Urbeui Institute, in the identi- 
fication of alternative instructional methodologies useful as intervention 
techniques; Lehigh University, in the validation of indicators of school effective 
ness; and the Pennsylvania State University, in the effort to improve predicta- 
bility equations. 

How caxi the results be used effectively? Other than the obvious comparisons 
of student performance with the state normative group, and with performance 
levels from schools operating within similar sets of conditions, the results can 
be used to: 

1. Detexrraine priorities of attack 

2. Provide focus for examining individual student test results 
which already exist 



3. Measure effectiveness of curriculum changes 

4. Provide objective data to support requests for programs, 
equipment, etc^ 

Evidence that the assessment findings are used to bring about educational 
change is reported by J. Robert Coldlron*' in his study of their utilization by 
101 participating local school districts. He found that 91% of district adminis- 
trators reviewed their current educ6itional programs in the light of assessment 
findings* Fifty-two of the 101 districts reported some program revisions, 
another 52 reported that their results enabled them to identify specific problem 
areas in student performance, staffing ]x>licies or organizational strucutre. 
Finally., 17% of the districts reported the Incorporation of totally new educa- 
tional programs as an outgrowth of the assessment findings. 

Statewide needs^ identified from the assessment findings, were found to be: 
Grade 3 - Interest in school ^uid learning 

Vocational Understanding 
Grade 11 - Interest in school and learning 

Reexamination of the data for different types of school communities provided 
evidence that the following needs existed in addition to those previously identi- 
fied tq be statewide. 



Urban 

Suburban 

Rural 



Grade 5 



Verbal Skills 



Verbal Skills 
Self Understanding 



Grade 11 

Verbal Skills 

Citizenship 

Verbal Skills 
Self Understanding 



* Coldiron, J. Robert. "An Investigation of the Utilization of Requested 

Assessment Information in Pennsylvania School Districts,'* Unpublished Doctor's 
Thesis, The Pennsylvania State University, University Park, 1973 
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A further analysis of the data on relationship to appalachlan and non- 
appalachlan areas of the Coranonwealth Identified verbal skills and appreciating 
human accompli shioent to be needs at the fifth grade level and the understanding 
of others who are different in racet religion or socioeconomic status for 
eleventh grade students attending schools located in app&lachian areas* 

Pennsylvania's State Board of Education passed a resolution in Kovember 
1973 requiring all schools to undergo assessment at grades 5, 8 and 11 within 
the next three years. ThiSa In effect, requires the, assessment of one-third 
of the local school districts eeu:h year until 1976. ^ The Department of Education 
will obtain a reliable estimate of student perfonsance statewide in 1974 and 
1975 ais well as complete assessment results in 1976.. Upon. completion of the 
first cycle a second eye le^ will be carried out in the period 1977^-1979 to 
provide both croris- sectional and psuedolongitudlnal data to aid in determining 
what changes in t:he levels of student performance have occurred. 



